This application is based upon and claims the benefit of priority from the prior Japanese Patent Application No. 2002-013001, filed on Jan. 22, 2002, the entire contents of which are incorporated herein by reference.
This invention relates to a system and method for processing an image and a compiler for use in this application, and particularly relates to a system and method capable of carrying out rapid data processing by reducing the number of loading/storing operations when processing a large amount of image data, and a compiler for use in this system.
In general, in the field of image processing technique, such as computer graphics, it is desired to provide an image processing system having a flexible image processing function in order to realize various expressions relating to shape, color and so forth in an image. Also with respect to a compiler serving as software for use in this image processing system, it is desired to provide a compiler capable of processing a large amount of data relating to shape, color and so forth in a colorful image.
For example, in a case where color is determined on the basis of a texture showing a pattern by brightness and color in an image, the color of the image is determined on the basis of coordinates of pixels forming the image. Also during this processing for determining color, a large amount of data must be used to be sequentially operated. For that reason, image processing systems (Graphic Processor Units (GPUs)) for carrying out rapid operation are conventionally used.
However, in conventional GPUs, it is possible to deal with only fixed data flows or semi-fixed data flows, and the processing of pixels capable of being executed at a time (by one path. For that reason, it is required to use a plurality of paths of GPUs in order to carry out a complicated operation. In a case where a plurality of paths of GPUs are used, after intermediate data is temporarily written in a memory, such as a frame buffer, in the last path, data written in the memory, such as the frame buffer, is read out of the memory in the subsequent path, so that the number of loading/storing operations increases with respect to the memory.
There is a problem in that the increase of the number of loading/storing operations with respect to the memory causes a bottle neck for the flow of data processing in the whole image processor to reduce image processing performance, so that improvements are desired from the standpoint of rapid data processing. For example, the GeForce 3 GPU produced by xe2x80x9cnVidia Companyxe2x80x9d has a programmable pixel processor, and is capable of programming an address operator and a combiner for blending pixel color and so forth, by a series of instruction strings, respectively (See Paper: GeForce 3 GPU [Compiling to a VLIW Fragment Pipeline: FIGS. 1, 2]).
However, also in the chip produced by nVidia Company disclosed in the above described paper, the number of instructions capable of being executed with respect to one pixel at a time is limited, and the number of registers is also limited. Therefore, it is required to use a plurality of paths of pixel processors to carry out a more complicated processing. The reason for this is that the address operator is separated from the combiner in the image processor serving as the prior art, so that it is required to separate into a plurality of paths when it is intended to carry out operation in order from the combiner to the address operator.
If it is possible to provide an image processing system for carrying out a more flexible processing by one path and a compiler for use therein, it is possible to reduce the number of paths in a pixel operation, and it is possible to reduce the number of loading/storing operation in memory which tends to be a bottle neck for performance.
An image processing system according to a first aspect of the present invention comprises: a plurality of operation pipelines to operate an inputted image data; a switching channel to switch a data transfer path to input operation results, which are outputted from the plurality of operation pipelines, to the plurality of operation pipeline again; and a control circuit to control switching of the data transfer path by the switching channel and to control an operation in the plurality of operation pipelines, the control circuit carrying out a scheduling of a plurality of operations, which form (nxe2x88x92k+1) unit operations from a unit operation k (1 less than k less than n) to a unit operation n (n is a positive integer) of unit operations 1 to n, the plurality of operations prevented from overlapping with each other at the same predetermined operation time in the same operation pipeline when a unit operation included in the plurality of operations is executed by the plurality of operation pipelines.
A compiler according to a second aspect of the present invention for use in an image processing system having a plurality of operation pipelines to operate an inputted image data; a switching channel to switch a data transfer path to input operation results, which are outputted from the plurality of operation pipelines, to the plurality of operation pipeline again; and a control circuit to control the switching of the data transfer path by the switching channel and to control an operation in the plurality of operation pipelines, the compiler comprising receiving a part of the image data supplied from the outside each operation pipeline included in the plurality of operation pipelines, and scheduling a plurality of operations, which form (nxe2x88x92k+1) unit operations from a unit operation k (1 less than k less than n) to a unit operation n (n is a positive integer) of n unit operations from 1 to n forming the plurality of operations, the plurality of operations prevented from overlapping with each other at the same operation time in the same operation pipeline, and controlling a sequence of operations in the plurality of operation pipelines in accordance with scheduling.
An image processing method according to a third aspect of the present invention comprises: supplying a part of inputted image data, which is supplied from the outside, to each operation pipeline forming a plurality of operation pipelines to operate the image data; carrying out a scheduling of a plurality of operations, which form (nxe2x88x92k+1) unit operations from a unit operation k (1 less than k less than n) to a unit operation n (n is a positive integer) of n unit operations from 1 to n forming the plurality of operations, the plurality of operations prevented from overlapping with each other at the same operation time in the same operation pipeline; and controlling the operation of the image data in accordance with the scheduling.